How the Cases You Choose Affect the Answers You Get: Selection Bias in Comparative Politics

This article demonstrates how the selection of cases for study on the basis of outcomes on the dependent variable biases conclusions. It first lays out the logic of explanation and shows how it is violated when only cases that have achieved the outcome of interest are studied. It then examines three wellknown and highly regarded studies in the field of comparative politics, comparing the conclusions reached in the original work with a test of the arguments on cases selected without regard for their position on the dependent variable. In each instance, conclusions based on the uncorrelated sample differ from the original conclusions. Comparative politics, like other subfields in political science, has norms and conventions about what constitutes an appropriate research strategy and what kind of evidence makes an argument persuasive. One of our most durable conventions is the selection of cases for study on the dependent variable. That is, if we want to understand something, for example, revolution, we select one or more occurrences and subject them to scrutiny. Most graduate students learn in the statistics courses forced upon them that selection on the dependent variable is forbidden, but few remember why, or what the implications of violating this taboo are for their own work. And so, comparativists often ignore or forget about it when carrying out or assessing nonquantitative comparative research. I am grateful to Chris Achen, Jim Caporaso, David Collier, Jeff Frieden, Miriam Golden, Jonathan Hartlyn, Robert Jackman, and John Zaller for comments on earlier versions of this article. 1. The subfield of comparative politics is not the only one bedeviled by problems with selection bias (compare Achen and Snidal 1989).